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The use of data to inform decisionmaking and practice at the school and district levels is now a common feature 
of reform efforts. Advances in districts’ technological capacities have produced data systems that allow a flow of 
data to and from schools, often to the point of creating an overwhelming flood of information. To make the flow 
of information usable, it is critical that it is accompanied by clear decision rules on how to organize, interpret, 
and act on the data and the information they provide. Increasingly, schools and districts are turning to indicators 
to narrow the flow of data to the factors that truly matter for student success. Although in principle anything 
that can be quantified can be used as an indicator, effective indicators provide a signal of a later outcome. 
Effective indicators can be used to identify students who are likely to achieve (or fail to achieve) an outcome, or 
to highlight actionable leverage points for altering that outcome.* For college readiness indicators, it becomes 
particularly important to identify actionable leverage points because the outcome they are intended to predict- 
college success-occurs after the students have left high school. In other words, identifying which students are 
likely to complete college is only a starting point. 


This resource is intended to guide district administrators as they consider what 
indicators to include in their indicator systems and how to focus their efforts to 
improve students’ college readiness. It describes four characteristics of effective 
indicators, drawing on the work of the University of Chicago Consortium on 
Chicago School Research (UChicago CCSR) with the Chicago Public Schools 


(CPS). CPS used evidence from research to guide the development of two separate 
indicator systems: the on-track indicator for high school graduation and the Free 
Application for Federal Student Aid (FAFSA) completion indicator system for 
college enrollment. These experiences provide valuable lessons for other districts as 
they develop their own college readiness indicator systems. 


Effective Indicators Are Part of a System 


The effectiveness of indicators in isolation is limited; they need to be a part of 
an indicator system if they are to change outcomes. An indicator system links 
indicators to action and includes tools for reporting indicators and tracking 
progress; supports for building the capacity of adults to access, understand, and act 
upon the indicators; and strategies, supports, and interventions for the students 
identified by the indicators. The power of an indicator system is that it can take 
messy, seemingly overwhelming problems (such as getting students to graduate 


from high school and persist in college) and focus attention on the factors that 
have the highest leverage for improving outcomes through school strategies and 
intervention, or on the students who are most in need of intervention. 

Embedded in an indicator system, indicators do more than merely provide 
information; they become a common point of reference for making decisions, 
focusing efforts, and monitoring progress. Effective indicator systems bring 


* Indicators can be used for other purposes such as tracking progress, holding schools accountable for 
improvement, or signaling priorities. This article focuses on indicators that are intended to predict a 
future outcome. 
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coherence to efforts and provide guidance to many people: district personnel, 
school administrators, individual teachers, students, and parents. It is therefore 
critical that the indicators provide targeted and relevant information and that 
careful consideration has been given to how practitioners, administrators, and 
community members will use the indicators to support students and schools. This 
requires an understanding of how the indicators will be used in data reporting 
systems, how they fit into district priorities, and what capacity the district has to 
provide interventions and support. Thus, building an effective indicator system 
requires focusing effort on high-leverage indicators while also attending to the 
structures in which the indicators will be used. 

District leadership should shape the priorities for the indicator system, but the 
process of selecting indicators and building the indicator system should not rest 


solely on the district leadership. Instead, these decisions and processes should 
be a collaborative effort involving all levels of the district so that the needs of 
multiple stakeholders in different stages of indicator use are taken into account. 
Data analysts can provide evidence of the validity of the indicators and ensure that 
the data yield accurate and timely information. Administrators of programs and 
interventions provide an important perspective on what kinds of information and 
resources are needed. Community members are part of the system of supports 
around students and give an important external perspective. Finally, school 
practitioners have a clear perspective of how the data are used on the ground and 
are ultimately the most direct link between the indicators and the student supports 
that can lead to improved outcomes. 


Four Characteristics of Effective Indicators 


The process of selecting indicators should be informed by the larger district 
context, who the end users will be, and how the information provided by 
the indicators will help guide users’ actions. This resource focuses on four 
characteristics of effective indicators and how they fit with larger efforts and 
systems of supports that provide a coherent roadmap for districts. Effective 
indicators are: 


Valid for the Intended Purpose 


Effective indicators are valid for the intended purpose, based on research evidence 
and, if possible, careful analysis of a district’s own data. Too often, variables are 
chosen as indicators because they are widely believed to matter for later outcomes, 
but they show no relationship to the outcomes when examined rigorously. For 
example, test scores are often used to determine which students need support to 
pass their classes and eventually graduate from high school. However, UChicago 
CCSR analyzed CPS data and found that 8th grade standardized tests scores 
showed only a modest relationship with course failure. 1 That is, using this indicator 
to identify students at risk of course failure would be extremely imprecise and 
lead to little change in the outcome that it is intended to improve — high school 
graduation. Although test scores have been shown to be only a weak predictor of 


1. valid for the intended purpose; 

2. actionable by schools; 

3. meaningful and easily understood by practitioners; and 

4. aligned with the priorities of the district and schools. 


course failure in Chicago, other districts should examine the relationship using 
their own data on student test scores and course performance. 

Further research at UChicago CCSR has shown student attendance to be a much 
better predictor of high school graduation than test scores. 2 Monitoring attendance 
allows practitioners to identify which students are likely to drop out and then work 
with those students to determine an individualized strategy for re-engaging them 
in school. Furthermore, by developing strategies to re-engage students with low 
attendance, practitioners can have a much larger effect on graduation than they 
would through raising test scores because the relationship between attendance and 
graduation is stronger and more direct than the relationship between test scores 
and graduation. 
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Actionable by Schools 


There is a long list of variables that predict high school dropout and college access 
and graduation, and many of them fall outside the domain of school and teacher 
influence. Indicators are actionable by schools when they provide guidance as 
to which students to target for intervention or which areas of college readiness 
to focus their efforts on. Effective indicators condense seemingly unmanageable 
problems into factors that educators can influence by changing students’ day-to-day 
experiences. For example, UChicago CCSR focused on what schools could do to 
reduce 9th grade course failures, an indicator of future high school dropout. 3 The 
initial finding that absences were the primary driver of course failure led to the 
subsequent finding that even one week of absences in the first quarter of 9th grade 
reduced the likelihood of a student graduating four years later by 15 percentage 
points — an actionable indicator that brought the problem of high school graduation 
down to students’ behavior and experiences in the first weeks of high school. 

When the purpose of an indicator is to identify leverage points for focusing a 
school’s efforts, it is also important that the indicators are malleable through school 
strategies and interventions. For example, knowing that students who were retained 
in elementary school are more likely to drop out of high school can help 9th grade 
educators identify which students are at risk. However, an indicator of elementary 
school retention does not provide high schools with a malleable leverage point for 
intervention because the students have already been retained by the time they reach 
high school. In contrast, effective indicators help practitioners identify leverage 


points within their control so that they can develop strategies and interventions to 
improve the intended outcome. 

When indicators measure something that a school is able to influence, efforts 
to monitor progress on the indicator have real meaning to practitioners. Often 
practitioners are frustrated when their efforts do not translate into improvements 
in the outcomes that were targeted. Sometimes this occurs because their efforts 
are aimed at indicators that are only modestly or spuriously related to the outcome 
they care about — these are issues of validity as described above. Other times this 
occurs because the intervention or strategy that is used is not appropriate for the 
students who are targeted for intervention. For example, high schools often target 
students with multiple course failures for academic interventions such as tutoring. 
UChicago CCSR’s analysis suggests that these students’ graduation rates will not 
be moved by modest interventions and instead require much more intensive help 
that may extend beyond academic support. 4 In contrast, targeting students with 
just one F to receive modest interventions may have a substantial impact on their 
graduation rates. Effective indicators help practitioners differentiate among students 
and target specific strategies to the students who are most likely to be helped by 
them. Moreover, effective indicators provide a means through which school-level 
practitioners can evaluate the potential impacts of their strategies on the outcomes 
they care about for different groups of students. 


Meaningful and Easily Understood by Practitioners 


The extent to which an indicator is valid and actionable must be balanced with the 
ease with which school staff can understand and act upon it. Districts must rely 
on the capacity of the adults in schools to respond to the information provided 
by indicators and translate that information into supports that are effective for 
improving student outcomes. Indicators that are overly complex or do not provide 
information that is pertinent to school practitioners are unlikely to improve the 
outcomes they are intended to address. 

Indicators need to be easily understood so that they can be translated into reporting 
tools. Effective reporting tools allow practitioners to intervene accordingly for 
students at risk and to develop targeted plans and appropriate guidance for all 


students to maximize their success and performance in high school and college. 

This also means that an indicator must be available and reported in a timely manner 
so that practitioners can intervene with students as needed. Effective indicators are 
also easily understood by students, their families, and the wider community so that 
they can support schools in their college readiness efforts. This form of reporting 
also provides the public with information that helps students, their families, and the 
community understand and assess schools and their progress on indicators. 

The freshman on-track indicator at CPS is an example of an indicator that has 
become an actionable leverage point, in part because it is meaningful and easy 
to understand. It is comprised of just two factors — whether a student has earned 
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at least five credits and whether the student has failed no more than one core 
course — that are familiar to school staff. As part of its freshman on-track indicator 
system, CPS developed a series of online tools meant to help school practitioners 
identify at-risk students, monitor individual students’ performance, and gauge their 
schools’ progress in supporting students’ transition to high school. The district has 
a “freshman watch list,” which includes students’ 8th grade attendance, grades, and 


Aligned with the Priorities of the District and Schools 


With the current deluge of information and competing priorities facing 
practitioners and administrators, simply providing more data in the form of 
indicators will not change practice. Indicators become effective when the district 
leadership prioritizes the indicators and related initiatives among their other goals 
and provides support and resources that help people understand how to use the 
indicators and integrate them into their practice on an ongoing basis. 

Effective indicators also provide a means of signaling the priorities of the district 
leadership to school practitioners, administrators, and the community. The 


Example: CPS’s FAFSA Tracking System 


The FAFSA Tracking System was one of the first ventures CPS made in developing 
an indicator system. It introduced the use of data in schools as a tool to develop 
strategies around a common problem and monitor their progress, rather than 
strictly as part of an accountability system. The indicator system included supports 
designed to build schools’ capacity to use data and to translate the information 
embedded in the data into appropriate action. 

Improving access to college was a priority of district leadership; in 2002, ensuring 
that all students graduate from high school and enroll in postsecondary education 
or training was one of the eight goals for CEO Arne Duncan’s administration. Two 
years later, the district established the Department of Postsecondary Education, 
which consolidated existing programs and resources around postsecondary access 
and introduced new resources such as postsecondary coaches and specialists. The 
department provided a home and clear leadership for the district’s postsecondary 
access efforts. Duncan also communicated the importance of FAFSA completion 
by sending weekly reports to high school principals with updates on the schools 
that had been making progress on FAFSA completion and those that were lagging 
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test scores and flags the students that are at risk of struggling in 9th grade. It has 
also developed “freshman success reports,” which track students’ 9th grade grades, 
attendance, and test scores and flags those students with course failures and/or high 
absence rates, helping practitioners target appropriate supports for students with 
different needs. This has allowed school practitioners to plan and closely monitor 
their students in the early stages of high school. 


indicator becomes a tangible point of reference for evaluating status and monitoring 
progress and can also serve to increase awareness of an issue, develop a common 
language, and focus the attention of school-level practitioners, administrators, and 
the school community. For example, when CPS administrators developed a data 
system to flag students who were at risk of being off-track for graduation, it signaled 
the importance of monitoring students’ course performance and attendance as 
they entered high school. It also reinforced the district leadership’s perspective that 
high school dropout was within schools’ control, rather than being due to external 
circumstances. 


behind. These reports created competition among principals to organize their 
schools to improve their FAFSA completion rates. 

At the same time that CPS launched the Department of Postsecondary Education, 
UChicago CCSR began a major research initiative on college access using CPS data. 
One of UChicago CCSR’s early findings was that FAFSA completion was a major 
barrier to college enrollment and that one of the distinguishing characteristics 
between the students who had been accepted to college and enrolled and students 
who had been accepted but did not enroll was whether they had completed a 
FAFSA. 5 UChicago CCSR shared this early finding with CPS as soon as it was clear 
that using FAFSA completion as an indicator was valid for the intended purpose of 
improving college enrollment. 

Once it was clear that FAFSA completion was a barrier to college access, CPS 
entered into an agreement with the Illinois Student Assistance Commission to 
receive real-time data on students’ FAFSA completion. The FAFSA indicator was 
meaningful and easily understood by practitioners; it simply signified whether a 
student had successfully completed a FAFSA. Teachers and counselors previously 
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had to rely on students’ self-reports of whether they had completed a FAFSA. 

CPS developed the web-based FAFSA tracking system that allowed practitioners 
to log in and see data on which students had completed the FAFSA and which 
had not, as well as students’ eligibility for Pell Grants and their Expected Family 
Contribution to college costs. The FAFSA tracking system also included timely 
setting-level data that allowed progress to be monitored at the school and subgroup 
levels. The district worked closely with users of the reporting tools to refine them 
until the information was accessible and clear. The district further developed 
annual reporting tools and school-level reports on postsecondary preparation 
and outcomes, including trends that helped highlight what was working and what 
students needed further assistance. 

The FAFSA indicator was also clearly seen by practitioners as malleable and 
actionable by schools. For the first time, counselors had reliable information about 
whether a student had completed the FAFSA successfully and could target their 
efforts around the students who needed assistance. However, at the time, the FAFSA 
was a complicated form, and many counselors lacked formal training in how to best 
assist students and their parents in its completion. While learning how to complete 
a form seems like a relatively straightforward task, improving FAFSA completion 

Conclusion 


Educators, district-level administrators, and states are calling for the identification 
of effective indicators, particularly in the areas of high school and college readiness. 
While research on data-driven improvement offers a number of lessons for school 
districts implementing early warning and college readiness indicator systems, the 
field still lacks a rigorous understanding of the specific practices around indicator 
development, linking indicators to actions, reporting on indicators, and capacity- 
building efforts that are most effective in changing how districts and schools use 
data to improve student outcomes. 

The success of CPS’s FAFSA tracking system is, unfortunately, an anomaly. 
Indicators are often selected and put into data systems with too little attention 
paid to how they fit into the larger picture of what is happening in schools and 
the district. The work of selecting indicators is not just a matter of providing 
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rates required strong support from the department in training counselors and 
partnering with local higher education and other organizations to run workshops 
for students and their parents. 

FAFSA completion was only the first step to improving college access, but it 
established a foundation for using data-informed strategies in schools. The efforts 
to increase FAFSA completion had dramatic effects; the percentage of students 
completing a FAFSA by April 1 nearly tripled, going from 30% to 86% between 2007 
and 2011. The greater focus on college access and the increase in FAFSA completion 
was accompanied by a 10 percentage point increase in college enrollment, from 
50% to 60%, over the same time period. Practitioners and administrators were able 
to use the information provided by the FAFSA indicator to direct their resources, 
and the district provided supports for schools to accomplish the task so that the 
indicator was part of a larger indicator system. The district leadership was able to 
use FAFSA completion as a way of communicating college enrollment as a priority, 
and it was able to monitor the progress of schools on a regular basis. Perhaps most 
relevant to other districts seeking to implement a college readiness indicator system, 
the FAFSA tracking system changed how practitioners approached problems by 
using an indicator to be more strategic about their actions. 


practitioners, administrators, and the community with valid measures. It is also 
important that the indicators are relevant to the priorities of the district and the 
central problems faced by schools. They must be actionable in schools. It is also 
essential that they are easily understood and translated into reporting mechanisms 
so that practitioners are able to link the indicators to action. Indicators can be a 
valuable tool for analyzing what is happening inside schools and to students, and 
they provide a means for discussing problems and identifying what strategies may 
be effective for different students and schools. This requires that indicators be 
viewed as a part of a larger system of supports, that schools develop the capacity and 
organizational culture around data use, and that the district provides schools with 
the guidance and resources for developing this capacity and culture. 
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